Facial Features for Affective State Detection in Learning Environments

This study investigated facial features to detect the affective states (or emotions) that accompany deep-level learning of conceptual material. Videos of the participants’ faces were captured while they interacted with AutoTutor on computer literacy topics. After the tutoring session, the affective states (boredom, confusion, delight, flow, frustration, and surprise) of the student were identified by the learner, a peer, and two trained judges. Participants’ facial expressions were coded by two independent judges using Ekman’s Facial Action Coding System. Correlational analyses indicated that specific facial features could segregate confusion, delight, and frustration form the baseline state of neutral, but boredom was indistinguishable from neutral. We discuss the prospects of automatically detecting these emotions on the basis of facial features that are highly diagnostic.